Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add language detector middleware and helpers #3787

Merged
merged 10 commits into from
Feb 6, 2025

Conversation

lord007tn
Copy link
Contributor

Closes #3785

Description

Add language detector middleware to support future i18n functionality in Hono applications.

Features

  • Detect language from multiple sources:
    • Query parameters
    • Cookies
    • Accept-Language header
    • URL path
  • Type-safe language access via c.get('language')
  • Configurable detection order
  • Support for language caching
  • Comprehensive test coverage

Usage

import { languageDetector } from 'hono/language'

app.use('*', languageDetector({
  supportedLanguages: ['en', 'ar', 'es'],
  fallbackLanguage: 'en'
}))

app.get('/', (c) => {
  const lang = c.get('language')
  return c.text(`Current language: ${lang}`)
})

### The author should do the following, if applicable

- [X] Add tests
- [X] Run tests
- [X] `bun run format:fix && bun run lint:fix` to format the code
- [X] Add [TSDoc](https://tsdoc.org/)/[JSDoc](https://jsdoc.app/about-getting-started) to document the code
…function

@lord007tn
Copy link
Contributor Author

if this got accepted am willing to add docs in the official website to let people know how to use it

Copy link

codecov bot commented Dec 31, 2024

Codecov Report

Attention: Patch coverage is 87.81513% with 29 lines in your changes missing coverage. Please review.

Project coverage is 90.69%. Comparing base (1e62912) to head (ecdbd5e).
Report is 31 commits behind head on next.

Files with missing lines Patch % Lines
src/middleware/language/language.ts 84.02% 27 Missing ⚠️
src/helper/accepts/accepts.ts 0.00% 1 Missing ⚠️
src/utils/accept.ts 98.50% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             next    #3787      +/-   ##
==========================================
- Coverage   91.71%   90.69%   -1.03%     
==========================================
  Files         160      165       +5     
  Lines       10181    10480     +299     
  Branches     2896     3086     +190     
==========================================
+ Hits         9338     9505     +167     
- Misses        842      974     +132     
  Partials        1        1              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@lord007tn
Copy link
Contributor Author

i forgot to mention the full use of the middleware options are:

app.use(
  '*',
  languageDetector({
    // Detection strategies and their order
    order: ['path', 'querystring', 'cookie', 'header'],

    // Supported languages and fallback
    supportedLanguages: ['en', 'ar'],
    fallbackLanguage: 'en',

    // Query parameter settings
    lookupQuerystring: 'lang', // ?lang=en

    // Path settings
    lookupFromPathIndex: 0, // /en/about -> takes 'en'

    // Cookie settings
    lookupCookie: 'user-language',
    caches: false, // Set to false to disable caching

    // Header settings
    lookupFromHeaderKey: 'accept-language',

    // Language code handling
    ignoreCase: true,
    convertDetectedLanguage: (lang: string) => {
      // Optional: Convert language codes
      // e.g., 'en-US' -> 'en'
      return lang?.split('-')[0]?.toLowerCase() ?? '';
    },

    // Debugging
    debug: process.env.NODE_ENV !== 'production',
  }),
);

Copy link
Contributor

@askorupskyy askorupskyy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I love it! This entirely solves the lang detection for me. Would also be cool to have a way to disable cookie/query detection altogether, or could this be done with order param?


const DEFAULT_OPTIONS: DetectorOptions = {
order: ['querystring', 'cookie', 'header'],
lookupQuerystring: 'lang',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
lookupQuerystring: 'lang',
lookupQueryString: 'lang',

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I love it! This entirely solves the lang detection for me. Would also be cool to have a way to disable cookie/query detection altogether, or could this be done with order param?

yep just provide the order: ['header'] and it will only detect the header

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yusukebe
Copy link
Member

yusukebe commented Jan 8, 2025

Thanks @lord007tn I'll review this later.

Hey @sor4chi What do you think of this feature?

Copy link
Contributor

@sor4chi sor4chi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the wonderful PR, I liked it. I have one suggestion, what do you think?

Also, I feel a bit of discomfort with the naming hono/language. Personally, I would prefer something like hono/language-detector, but I’ll leave it to @yusukebe.

src/middleware/language/language.ts Show resolved Hide resolved
@yusukebe yusukebe changed the title add language detector middleware and helpers feat: add language detector middleware and helpers Jan 9, 2025
@yusukebe yusukebe added the v4.7 label Jan 9, 2025
@yusukebe
Copy link
Member

Hi @lord007tn

Sorry for my super late response! Let's prepare to ship this middleware in the next minor version, v4.7.

@sor4chi

I also have concerns regarding the name and whether it should be hono/language-detector or hono/language. I know language is too general, but if the function imported from hono/language is languageDetector, the user can imagine the feature, so I think hono/language is ok. This means, as @lord007tn wrote above:

import { languageDetector } from 'hono/language'

@yusukebe
Copy link
Member

@lord007tn

Can you apply the following diff to unify the coding style (using const for the function) with other code?

diff --git a/src/middleware/language/language.ts b/src/middleware/language/language.ts
index c298d3e9..e21f5a07 100644
--- a/src/middleware/language/language.ts
+++ b/src/middleware/language/language.ts
@@ -70,7 +70,7 @@ const DEFAULT_OPTIONS: DetectorOptions = {
  * @param header Accept-Language header string
  * @returns Array of parsed languages with quality scores
  */
-export function parseAcceptLanguage(header: string): Array<{ lang: string; q: number }> {
+export const parseAcceptLanguage = (header: string): Array<{ lang: string; q: number }> => {
   try {
     return header
       .split(',')
@@ -97,10 +97,10 @@ export function parseAcceptLanguage(header: string): Array<{ lang: string; q: nu
  * @param options Detector options
  * @returns Normalized language code or undefined
  */
-export function normalizeLanguage(
+export const normalizeLanguage = (
   lang: string | null | undefined,
   options: DetectorOptions
-): string | undefined {
+): string | undefined => {
   if (!lang) {
     return undefined
   }
@@ -126,7 +126,7 @@ export function normalizeLanguage(
 /**
  * Detects language from query parameter
  */
-export function detectFromQuery(c: Context, options: DetectorOptions): string | undefined {
+export const detectFromQuery = (c: Context, options: DetectorOptions): string | undefined => {
   try {
     const query = c.req.query(options.lookupQuerystring)
     return normalizeLanguage(query, options)
@@ -138,7 +138,7 @@ export function detectFromQuery(c: Context, options: DetectorOptions): string |
 /**
  * Detects language from cookie
  */
-export function detectFromCookie(c: Context, options: DetectorOptions): string | undefined {
+export const detectFromCookie = (c: Context, options: DetectorOptions): string | undefined => {
   try {
     const cookie = getCookie(c, options.lookupCookie)
     return normalizeLanguage(cookie, options)
@@ -238,7 +238,7 @@ function cacheLanguage(c: Context, language: string, options: DetectorOptions):
 /**
  * Detect language from request
  */
-function detectLanguage(c: Context, options: DetectorOptions): string {
+const detectLanguage = (c: Context, options: DetectorOptions): string => {
   let detectedLang: string | undefined

   for (const detectorName of options.order) {
@@ -277,7 +277,7 @@ function detectLanguage(c: Context, options: DetectorOptions): string {
  * @param userOptions Configuration options for the language detector
  * @returns Hono middleware function
  */
-export function languageDetector(userOptions: Partial<DetectorOptions> = {}): MiddlewareHandler {
+export const languageDetector = (userOptions: Partial<DetectorOptions>): MiddlewareHandler => {
   const options: DetectorOptions = {
     ...DEFAULT_OPTIONS,
     ...userOptions,
@@ -289,7 +289,7 @@ export function languageDetector(userOptions: Partial<DetectorOptions> = {}): Mi

   validateOptions(options)

-  return async function languageDetectorMiddleware(c, next) {
+  return async function languageDetector(c, next) {
     try {
       const lang = detectLanguage(c, options)
       c.set('language', lang)

@lord007tn
Copy link
Contributor Author

i just pushed new parse-accept helpers
and unified the logic in accept and language

my question now is for v4.7 we merge this and then turn to the docs or I do the docs to get this merged?

@sor4chi
Copy link
Contributor

sor4chi commented Jan 23, 2025

If you submit a PR to the honojs/website and include a link to this PR, @yusukebe should merge it at the same time as the release.

Copy link
Contributor

@sor4chi sor4chi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very good! Thank you for your support.
I have noted a few other points of concern.

src/middleware/language/index.ts Outdated Show resolved Hide resolved
src/utils/parse-accept.ts Outdated Show resolved Hide resolved
Comment on lines 53 to 61
if (qVal === undefined) {
return 1.0
}
if (qVal === '') {
return 1
}
if (qVal === 'NaN') {
return 0
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

q: is there any reason for using "1.0" and "0"?

language: string
}

const DEFAULT_OPTIONS: DetectorOptions = {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be better to export this.

@lord007tn
Copy link
Contributor Author

Very good! Thank you for your support. I have noted a few other points of concern.

i fixed some points you raised.

Copy link
Contributor

@sor4chi sor4chi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thank you very much!

@lord007tn
Copy link
Contributor Author

If you submit a PR to the honojs/website and include a link to this PR, @yusukebe should merge it at the same time as the release.

I’ve added the documentation honojs/website#569 to the honojs website. Since I’m not a technical writer by trade, please feel free to suggest improvements or refine the content for clarity and accuracy.

src/utils/parse-accept.ts Outdated Show resolved Hide resolved
src/utils/parse-accept.test.ts Outdated Show resolved Hide resolved
@yusukebe
Copy link
Member

yusukebe commented Feb 3, 2025

Hi @lord007tn

Almost all are good. I've left comments. Check them!

src/middleware/language/index.ts Outdated Show resolved Hide resolved
@yusukebe yusukebe changed the base branch from main to next February 6, 2025 13:08
@yusukebe
Copy link
Member

yusukebe commented Feb 6, 2025

Hi @lord007tn

Looks good to me! I'll merge this into the next branch for the next minor version and ship the v4.7 soon! Thank you for the great work!

@yusukebe yusukebe merged commit fa92596 into honojs:next Feb 6, 2025
16 checks passed
alexandresoro pushed a commit to alexandresoro/ouca-backend that referenced this pull request Feb 8, 2025
This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [hono](https://hono.dev) ([source](https://github.com/honojs/hono)) | dependencies | minor | [`4.6.20` -> `4.7.0`](https://renovatebot.com/diffs/npm/hono/4.6.20/4.7.0) |

---

### Release Notes

<details>
<summary>honojs/hono (hono)</summary>

### [`v4.7.0`](https://github.com/honojs/hono/releases/tag/v4.7.0)

[Compare Source](honojs/hono@v4.6.20...v4.7.0)

### Release Notes

Hono v4.7.0 is now available!

This release introduces one helper and two middleware.

-   Proxy Helper
-   Language Middleware
-   JWK Auth Middleware

Plus, Standard Schema Validator has been born.

Let's look at each of these.

#### Proxy Helper

We sometimes use the Hono application as a reverse proxy. In that case, it accesses the backend using `fetch`. However, it sends an unintended headers.

```ts
app.all('/proxy/:path', (c) => {
  // Send unintended header values to the origin server
  return fetch(`http://${originServer}/${c.req.param('path')}`)
})
```

For example, `fetch` may send `Accept-Encoding`, causing the origin server to return a compressed response. Some runtimes automatically decode it, leading to a `Content-Length` mismatch and potential client-side errors.

Also, you should probably remove some of the headers sent from the origin server, such as `Transfer-Encoding`.

[Proxy Helper](https://hono.dev/docs/helpers/proxy) will send requests to the origin and handle responses properly. The above headers problem is solved simply by writing as follows.

```ts
import { Hono } from 'hono'
import { proxy } from 'hono/proxy'

app.get('/proxy/:path', (c) => {
  return proxy(`http://${originServer}/${c.req.param('path')}`)
})
```

You can also use it in more complex ways.

```ts
app.get('/proxy/:path', async (c) => {
  const res = await proxy(
    `http://${originServer}/${c.req.param('path')}`,
    {
      headers: {
        ...c.req.header(),
        'X-Forwarded-For': '127.0.0.1',
        'X-Forwarded-Host': c.req.header('host'),
        Authorization: undefined,
      },
    }
  )
  res.headers.delete('Set-Cookie')
  return res
})
```

Thanks [@&#8203;usualoma](https://github.com/usualoma)!

#### Language Middleware

[Language Middleware](https://hono.dev/docs/middleware/builtin/language) provides 18n functions to Hono applications. By using the `languageDetector` function, you can get the language that your application should support.

```ts
import { Hono } from 'hono'
import { languageDetector } from 'hono/language'

const app = new Hono()

app.use(
  languageDetector({
    supportedLanguages: ['en', 'ar', 'ja'], // Must include fallback
    fallbackLanguage: 'en', // Required
  })
)

app.get('/', (c) => {
  const lang = c.get('language')
  return c.text(`Hello! Your language is ${lang}`)
})
```

You can get the target language in various ways, not just by using `Accept-Language`.

-   Query parameters
-   Cookies
-   `Accept-Language` header
-   URL path

Thanks [@&#8203;lord007tn](https://github.com/lord007tn)!

#### JWK Auth Middleware

Finally, middleware that supports JWK (JSON Web Key) has landed. Using [JWK Auth Middleware](https://hono.dev/docs/middleware/builtin/jwk), you can authenticate by verifying JWK tokens. It can access keys fetched from the specified URL.

```ts
import { Hono } from 'hono'
import { jwk } from 'hono/jwk'

app.use(
  '/auth/*',
  jwk({
    jwks_uri: `https://${backendServer}/.well-known/jwks.json`,
  })
)

app.get('/auth/page', (c) => {
  return c.text('You are authorized')
})
```

Thanks [@&#8203;Beyondo](https://github.com/Beyondo)!

#### Standard Schema Validator

[Standard Schema](https://standardschema.dev/) provides a common interface for TypeScript validator libraries. [Standard Schema Validator](https://github.com/honojs/middleware/tree/main/packages/standard-validator) is a validator that uses it. This means that Standard Schema Validator can handle several validators, such as Zod, Valibot, and ArkType, with the same interface.

The code below really works!

```ts
import { Hono } from 'hono'
import { sValidator } from '@&#8203;hono/standard-validator'
import { type } from 'arktype'
import * as v from 'valibot'
import { z } from 'zod'

const aSchema = type({
  agent: 'string',
})

const vSchema = v.object({
  slag: v.string(),
})

const zSchema = z.object({
  name: z.string(),
})

const app = new Hono()

app.get(
  '/:slag',
  sValidator('header', aSchema),
  sValidator('param', vSchema),
  sValidator('query', zSchema),
  (c) => {
    const headerValue = c.req.valid('header')
    const paramValue = c.req.valid('param')
    const queryValue = c.req.valid('query')
    return c.json({ headerValue, paramValue, queryValue })
  }
)

const res = await app.request('/foo?name=foo', {
  headers: {
    agent: 'foo',
  },
})

console.log(await res.json())
```

Thanks [@&#8203;muningis](https://github.com/muningis)!

#### New features

-   feat(helper/proxy): introduce proxy helper honojs/hono#3589
-   feat(logger): include query params honojs/hono#3702
-   feat: add language detector middleware and helpers honojs/hono#3787
-   feat(hono/context): add buffer returns honojs/hono#3813
-   feat(hono/jwk): JWK Auth Middleware honojs/hono#3826
-   feat(etag): allow for custom hashing methods to be used to etag honojs/hono#3832
-   feat(router): support greedy matches with subsequent static components honojs/hono#3888

#### All changes

-   docs(CONTRIBUTING): remove text about `yarn` by [@&#8203;EdamAme-x](https://github.com/EdamAme-x) in honojs/hono#3878
-   refactor(request): `toLowerCase()` is unnecessary for `req.header()` by [@&#8203;yusukebe](https://github.com/yusukebe) in honojs/hono#3880
-   fix(helper/adapter): correct `env` type by [@&#8203;yusukebe](https://github.com/yusukebe) in honojs/hono#3885
-   chore(test): update to vitest 3 by [@&#8203;yasuaki640](https://github.com/yasuaki640) in honojs/hono#3861
-   fix(router/trie-router): fix label with trailing wildcard pattern by [@&#8203;usualoma](https://github.com/usualoma) in honojs/hono#3892
-   feat(helper/proxy): introduce proxy helper by [@&#8203;usualoma](https://github.com/usualoma) in honojs/hono#3589
-   feat(logger): include query params by [@&#8203;ryuapp](https://github.com/ryuapp) in honojs/hono#3702
-   feat(factory): Allow HonoOptions<E> with factory by [@&#8203;miyaji255](https://github.com/miyaji255) in honojs/hono#3786
-   feat: add language detector middleware and helpers by [@&#8203;lord007tn](https://github.com/lord007tn) in honojs/hono#3787
-   feat(hono/context): add buffer returns by [@&#8203;askorupskyy](https://github.com/askorupskyy) in honojs/hono#3813
-   feat(hono/jwk): JWK Auth Middleware by [@&#8203;Beyondo](https://github.com/Beyondo) in honojs/hono#3826
-   feat(etag): allow for custom hashing methods to be used to etag by [@&#8203;EdamAme-x](https://github.com/EdamAme-x) in honojs/hono#3832
-   feat(router): support greedy matches with subsequent static components. by [@&#8203;usualoma](https://github.com/usualoma) in honojs/hono#3888
-   fix(client): correct inferring empty object from`c.json({})` by [@&#8203;yusukebe](https://github.com/yusukebe) in honojs/hono#3873
-   Next by [@&#8203;yusukebe](https://github.com/yusukebe) in honojs/hono#3896
-   chore(runtime-tests): add `deno.lock` by [@&#8203;yusukebe](https://github.com/yusukebe) in honojs/hono#3897

#### New Contributors

-   [@&#8203;lord007tn](https://github.com/lord007tn) made their first contribution in honojs/hono#3787

**Full Changelog**: honojs/hono@v4.6.20...v4.7.0

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4xNjIuMiIsInVwZGF0ZWRJblZlciI6IjM5LjE2Mi4yIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJkZXBlbmRlbmNpZXMiXX0=-->

Reviewed-on: https://git.tristess.app/alexandresoro/ouca-backend/pulls/537
Reviewed-by: Alexandre Soro <code@soro.dev>
Co-authored-by: renovate <renovate@git.tristess.app>
Co-committed-by: renovate <renovate@git.tristess.app>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Language Detector Middleware for Hono
4 participants